Micro-batch and data frequency for stream processing on multi-cores

Latency or throughput is often critical performance metrics in stream processing. Applications’ can fluctuate depending on the input stream. This unpredictability due to variety data arrival frequency and size, complexity, other factors. Researchers are constantly investigating new ways mitigate impact of these variations with self-adaptive techniques involving elasticity micro-batching. However, there a lack benchmarks capable creating test scenarios further evaluate techniques. work extends improves SPBench benchmarking framework support dynamic micro-batching management. We also propose set algorithms that generates most commonly used patterns for processing related work. It allows creation wide scenarios. To validate our solution, we use create custom Intel TBB FastFlow. These two libraries leverage parallelism multi-core architectures. Our results demonstrated cases did not benefit from micro-batches multi-cores. For different configurations, ensured lowest latency, while FastFlow assured higher shorter pipelines.